Estimation of Covariance Matrices under Sparsity Constraints

Discussion of “Minimax Estimation of Large Covariance Matrices under L1-Norm” by Tony Cai and Harrison Zhou. To appear in Statistica Sinica. Introduction. Estimation of covariance matrices in various norms is a critical issue that finds applications in a wide range of statistical problems, and especially in principal component analysis. It is well known that, without further assumptions, the empirical covariance matrix Σ is the best possible estimator in many ways, and in particular in a minimax sense. However, it is also well known that Σ is not an accurate estimator when the dimension p of the observations is high. The minimax analysis carried out by Tony Cai and Harry Zhou ([CZ] in what follows) guarantees that for several classes of matrices with reasonable structure (sparse or banded matrices), the fully data-driven thresholding estimator achieves the best possible rates when p is much larger than the sample size n. This is done, in particular, by proving minimax lower bounds that ensure that no estimator can perform better than the hard thresholding estimator, uniformly over the sparsity classes Gq for each 0 ≤ q < 1. This result has a flavor of universality in the sense that one and the same estimator is minimax optimal for several classes of matrices. Our comments focus on the sparsity classes of matrices. (a) Optimal rates. Optimal rates are obtained in [CZ] under the assumption that the dimension is very high: p ≥ nν, ν > 1. Thus, the case of dimensions smaller than n, or even p ≈ n, is excluded. This seems to be due to the technique of proving the lower bound (Theorem 2 in [CZ]). Indeed, by a different technique, we show that the lower bound holds without this assumption, cf. Theorem 1 below. Furthermore, in general, our lower rate ψ(1) is different from that obtained in [CZ] and has ingredients similar to the optimal rate for the Gaussian sequence model. We conjecture that it is optimal for all admissible configurations of n, p, and sparsity parameters. (b) Frobenius norm and global sparsity. We argue that the Frobenius norm is naturally adapted to the structure of the problem, at least for Gaussian observations, and we derive optimal rates under the Frobenius risk and global sparsity assumption. Department of Operations Research and Financial Engineering, Princeton University, Princeton, NJ 08544, USA. Supported in part by NSF grants DMS-0906424 and CAREER-DMS-1053987. Laboratoire de Statistique, CREST-ENSAE, 3, av. Pierre Larousse, F-92240 Malakoff Cedex, France. Supported in part by ANR under the grant “Parcimonie”.